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INTRODUCTION 

Telepresence  and  the  Transparent  Interface 

The  fundamental  purpose  of  a  telepresence  system  is  to  extend  an 
operator’s  sensory -motor  facilities  and  problem  solving  abilities  to  a  remote 
environment  [15].  Telepresence  is  achieved  by  projecting  the  operator's 
manipulatory  dexterity  to  a  remote  environment  while  reflecting  sensory 
feedback  so  realistically  that  the  operator  feels  present  in  the  remote  site  [1], 
To  enhance  operator  performance  and  understanding  within  remote 
environments,  most  research  and  development  of  telepresence  systems  has 
been  directed  towards  increasing  the  transparency  of  the  link  between 
operator  and  environment.  Much  of  this  work  has  focused  upon  improving 
the  fidelity  and  presentation  of  the  reflected  sensory  information.  Other  work 
has  focused  upon  making  the  projection  of  the  operator's  kinematics  and 
dynamics  more  naturally  mapped  into  the  remote  environment  [4,  5,  6,  9,  10]. 
The  culmination  of  such  research  efforts  could  be  a  transparent  link  between 
human  and  machine;  a  user  interface  through  which  information  is  passed  so 
naturally  between  operator  and  environment  that  the  user  achieves  a 
complete  sense  of  presence  within  the  remote  site. 

Although  unattainable  by  current  technology,  the  linking  of  an 
operator  to  a  remote  environment  through  a  perfectly  transparent  interface 
should  yield  a  human-machine  system  that  allows  an  operator  to  perform  just 
as  well  in  the  remote  environment  as  in  a  real  environment.  Although  the 
achievement  of  such  an  interface  would  be  an  important  milestone  in  human- 
machine  systems,  the  motivation  for  the  work  described  in  this  paper  is 
somewhat  different.  Rather  than  looking  for  methods  of  enhancing  the 
transparency  of  the  interface,  this  work  is  actually  focused  upon  corrupting 
the  transparency  of  the  interface  by  introducing  abstract  perceptual 
information  between  human  and  machine.  It  is  believed  that  abstract 
percepts  overlaid  on  top  of  the  reflection  of  the  workspace  can  be 
implemented  as  perceptual  tools  that  enhance  an  operator's  performance  and 
understanding.  It  is  not  suggested  that  such  an  approach  is  a  replacement  for 
a  high  fidelity  link  between  operator  and  environment,  but  rather  a 
supplement.  The  culmination  of  work  along  these  lines  could  be  a 
telepresence  system  which  does  more  than  simply  project  an  operator's 
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abilities  into  a  workspace,  but  actually  enhances  and  expands  the  operator's 
abilities  within  that  workspace  beyond  natural  capabilities. 

The  Concept  of  Perceptual  Overlays 

When  asked  to  draw  a  straight  line  in  the  real  world,  human 
performance  can  be  greatly  enhanced  by  using  a  simple  tool  such  as  a  ruler. 
The  use  of  a  ruler  reduces  the  amount  of  mental  processing  required  to 
perform  the  task,  speeds  an  operator's  line  drawing  ability,  and  most  of  all 

allows  an  operator  to  draw  a  significantly  better  line  than  if  no  ruler  had  been 
used  at  all.  Without  a  ruler,  line  drawing  is  a  manual  task  that  requires 

constant  visual  supervision  and  hand/eye  coordination.  With  a  ruler,  line 
drawing  is  not  only  faster  and  straighter,  but  the  dependance  on  visual 
feedback  is  reduced,  freeing  up  that  modality  for  other  uses.  What  is  more,  a 
ruler  is  often  used  as  a  barrier  to  protect  against  dangerous  or  destructive 
failures,  protecting  the  work-piece  from  the  slip  of  a  pencil  or  knife.  Such 

guidance  and  protection  allows  the  operator  to  ease  mental  criteria  for  task 
success  and  failure,  reducing  the  level  of  concentration  devoted  to  the  task. 

Although  a  simple  tool  by  any  standard,  a  common  ruler  is  clearly  a  powerful 
performance  aid  in  manual  line  drawing  tasks. 

Although  the  use  of  a  ruler  to  assist  in  straight-line  drawing  is  an 
effective  means  of  enhancing  human  performance,  can  such  a  process  be 
generalized  beyond  line  drawing?  Ruler-use  can  be  thought  of  as  nothing 
more  than  a  process  of  overlaying  abstract  sensory  information  on  top  of  a 
workspace.  Thus,  a  ruler  can  be  generalized  as  a  particular  “perceptual 
overlay”  designed  to  enhance  line  drawing  performance.  In  the  case  of  a 
ruler,  the  overlaid  sensory  information  represents  a  rigid  surface  that  is 
perceived  haptically  and  visually  by  the  user.  By  overlaying  this  generic 
piece  of  sensory  information  on  top  of  the  workspace,  the  user  has  reduced  the 
mental  and  physical  demands  of  the  straight  line  drawing  task  and 
performance  is  greatly  enhanced. 

If  a  simple  ruler-like  perceptual  overlay  can  so  greatly  enhance  the 
performance  of  real  world  manipulatory  tasks  like  straight  line  drawing,  it 
seems  that  computer  generated  perceptual  overlays  could  be  developed  within 
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virtual  environments  to  enhance  the  performance  of  tele-manipulation  tasks 

within  remote  worksites.  Just  as  a  ruler  can  be  overlaid  on  top  of  a  real 
workspace,  such  virtual  perceptual  overlays  could  be  overlaid  on  top  of  the 
sensory  feedback  from  a  remote  workspace. 

The  Virtual  Fixture  Metaphor 

Because  the  abstract  notion  of  overlaid  sensory  information  is  as 
difficult  to  conceptualize  as  it  is  to  talk  about,  I  will  introduce  a  virtual  fixture 
metaphor  as  a  means  of  describing  such  computer  generated  sensations  as 
concrete  physical  structures.  It  must  be  stressed  that  the  point  of  this 
metaphor  is  intended  to  facilitate  the  understanding  of,  and  interaction  with 

perceptual  overlays  and  should  not  be  taken  so  literally  as  to  limit  the  scope  of 
the  perceptual  overlay  concept.  Virtual  fixtures  are  thus  defined  as  abstract 
sensory  information  overlaid  on  top  of  reflected  sensory  feedback  from  a 

remote  environment.  Although  overlaid  on  top  of  the  user’s  perception  of  the 
remote  environment,  virtual  fixtures  are  completely  independent  of  all 
information  from  the  remote  site  and  are  thus  immune  from  communication 

delays  and  bandwidth  limitations. 

Like  the  ruler  guiding  the  pencil,  virtual  fixtures  overlaid  on  top  of  a 
remote  workspace  could  act  to  reduce  mental  processing  required  to  perform 

the  task,  reduce  the  work  load  of  certain  sensory  modalities,  and  most  of  all 
allow  precision  and  performance  to  exceed  natural  human  abilities.  Although 
virtual  fixtures  could  be  functionally  equivalent  to  fixtures  in  the  real  world, 
there  are  many  advantages  inherent  to  virtual  fixtures  because  they  are 

computer  simulations  rather  than  real  physical  hardware.  When  overlaid  on 

a  workspace,  the  fixtures  only  interact  with  the  user  and  not  with  the 

workspace  itself.  Thus  fixtures  can  occupy  the  same  physical  space  as  objects 

in  the  workspace.  This  means  that  the  workspace  geometry  imposes  no 
constraints  upon  the  placement  or  configuration  of  virtual  fixtures.  What  is 

more,  such  fixturing  has  no  mass,  has  no  physical  or  mechanical  constraints, 
requires  no  machining  time  or  maintenance,  can  be  easily  prototyped  and 

modified,  and  can  essentially  be  transported  to  remote  locations  using  nothing 

more  than  standard  communication  links. 
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If  we  explore  the  concept  of  virtual  fixtures  using  the  simple  ruler 
example  as  the  starting  point,  the  first  elements  to  consider  might  be  rigid 
planar  surfaces.  Such  fixtures  would  be  composed  of  haptic  sensations 
generated  by  reflecting  simulated  forces  to  the  operator  through  a  force- 
reflecting  master.  As  the  operator  interacts  with  the  modeled  surfaces,  the 
reaction  forces  would  be  computed  and  reflected  appropriately.  Of  course, 
such  fixtures  are  by  no  means  limited  to  rigid  surfaces.  Abstracting  the 
fixturing  concept,  we  might  consider  modeling  compliant  surfaces,  damped 
surfaces,  even  viscous  or  coulomb  frictional  contacts.  In  fact,  the  simulation 
environment  offers  such  freedom  that  fixtures  could  even  be  developed  as 
attractive  or  repulsive  fields.  Although  fixtures  composed  of  haptic  sensations 
offer  endless  possibilities,  the  fixturing  concept  is  not  limited  to  that  modality. 
Abstract  fixtures  could  be  composed  of  visual,  auditory,  even  tactile  sensations 
used  alone  or  in  cross-modal  combinations.  For  example,  if  a  haptic  fixture 
composed  of  rigid  planar  surfaces  or  attractive  force  fields  was  developed  to 
aid  a  particular  task,  an  audio,  visual,  or  vibratory  signal  could  be  mapped  to 
various  locations  along  the  fixture  to  enhance  interaction.  Such  additional 
modalities  could  be  used  to  indicate  deviations  from  a  trajectory,  proximity  to  a 
danger  zone,  even  provide  feedback  of  velocities  or  accelerations. 

Abstracting  the  fixturing  concept  further,  we  could  imagine  virtual 
fixtures  imbued  with  particular  visual  qualities  to  enhance  interaction  with 
environments.  For  example,  virtual  fixtures  composed  of  haptic  surfaces  could 
be  modeled  with  optical  properties  to  suit  the  task  at  hand.  The  fixture  might 
be  invisible  to  the  user  if  the  operator  gains  no  benefit  from  visual  cues,  it 
could  be  made  to  look  like  a  solid  object  if  rich  visual  cues  are  useful  for  the 
task,  it  could  even  be  made  to  look  like  a  transparent  glassy  solid  if  visual  cues 
are  important  but  the  user  wants  to  avoid  obscuring  the  workspace.  Fixtures 
might  even  be  designed  as  visual  filters  to  block  particular  distractions, 
enhance  contrast,  provide  depth  cues,  even  magnify  a  part  of  the  workspace. 
One  can  even  imagine  the  benefit  of  a  fixture  composed  of  compliant  surfaces 
which  changes  color  or  brightness  with  compression.  Previous  work  with 
targeting  cues  [3],  predictive  displays  [2,  14,  20,  21,  22],  and  perspective 
overlays  [11,  25]  has  demonstrated  that  overlaid  visual  cues  can  enhance 
performance  and  understanding  within  a  teleoperation  environment. 
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If  the  description  of  virtual  fixtures  thus  far  seems  too  abstract,  a  simple 
example  may  drive  the  concept  home.  Imagine  a  situation  where  a 

teleoperating  surgeon  performs  a  delicate  procedure  on  a  patient.  Although 
such  uses  of  telerobotics  are  still  in  the  research  phase,  it  is  an  application 
that  demands  a  high  degree  of  human  performance  within  a  remote 

workspace.  Now  imagine  that  a  virtual  fixture  is  being  used  by  the  doctor  to 
enhance  his  abilities  in  this  procedure.  The  fixture  might  appear  to  the  doctor 
like  a  flat  plane  of  glass  with  a  grooved  guide  for  the  scalpel.  The  glass-like 
virtual  fixture  might  actually  pass  directly  through  a  patient’s  body, 
preventing  the  scalpel  from  penetrating  below  a  particular  depth  but  not 
obscuring  vision  of  the  tissue  below.  By  sliding  the  scalpel  along  the  edge  of  a 
groove  in  the  fixture,  the  surgeon  could  make  a  perfect  incision.  The  slightest 
deviation  from  the  target  trajectory  might  be  reported  by  an  audio  or  tactile 

signal.  The  power  and  flexibility  of  such  a  system  would  be  unmatched  by 
actual  physical  tooling.  Besides  the  fact  that  such  a  fixture  in  the  real  world 

could  not  pass  directly  through  a  patient's  body,  it  could  not  be  put  in  place  at 
the  touch  of  a  button,  removed  at  the  touch  of  another  button,  or  easily  altered 
as  conditions  change.  What  is  more,  virtual  fixturing  does  not  have  to  be 
fabricated,  sterilized,  monitored,  or  maintained. 

Perceptual  Workstation  Environment 

Although  the  notion  of  virtual  fixturing  does  seem  promising,  if  the 
development  of  fixtures  requires  complex  computation  or  intimate  knowledge 
of  the  workspace  to  be  effective,  robot  autonomy  might  be  a  preferred  solution 
to  the  task  at  hand.  If,  on  the  other  hand,  effective  fixtures  could  be  developed 
out  of  basic  building  blocks  and  quickly  implemented  by  a  teleoperator  in  an 
interactive  environment,  fixtures  could  be  used  in  unstructured  or  changing 
environments  unsuitable  for  autonomous  systems.  Thus,  it  is  proposed  that  a 
workstation  type  environment  be  developed  to  allow  teleoperators  to  design 
and  implement  assistive  fixtures  when  confronted  with  an  unknown  task  in 
an  unstructured  environment  upon  first  encounter.  With  such  a  workstation 
in  mind,  the  study  described  in  this  paper  investigates  the  use  of  simple  haptic 
surfaces  and  auditory  fields  as  perceptual  aids  in  a  simple  peg  insertion  task. 
Although  a  workstation  environment  could  implement  more  sophisticated 
surfaces  or  fields  including  a  more  diverse  array  of  sensory  modalities,  it  was 
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thought  that  if  simple  combinations  of  forces  and  sounds  could  be  made  into 
effective  fixtures,  the  potential  of  virtual  fixtures  would  be  adequately 
displayed. 


Fitts'  Law  Performance  Test 

To  quantify  teleoperator  performance  in  a  remote  manual  task,  a  Fitts' 
Law  paradigm  was  chosen  because  of  its  general  acceptance  as  a  robust 
measure  of  human  performance  [7],  Although  extensive  use  of  Fitts'  Law  has 
been  documented  in  human  performance  literature,  little  work  has  been  done 
to  extend  the  paradigm  to  a  telepresence  environment.  McGovern  (1975)  used 
a  Fitts’  Law  task  to  demonstrate  the  merit  of  a  closed  loop  master-slave  system 
as  compared  to  an  open  loop  system  [13].  Hill  (1979)  used  a  Fitts’  task  to 
demonstrate  performance  differences  as  a  function  of  force  feedback  from  the 
manipulative  system  [8].  Pepper  (1988)  was  the  first  to  use  the  Fitts’  task  in  a 
true  telepresence  scenario.  While  previous  work  had  employed  Fitts’  Law  with 
the  teleoperator  in  direct  view  of  the  workspace,  this  work  used  Fitts’  Law  to 
compare  a  variety  of  viewing  conditions  which  included  remote  visual  links 
between  operator  and  workspace  [16].  These  studies  have  shown  that  a  Fitts’ 
Law  paradigm  is  appropriate  for  analysis  of  perceptual-motor  performance 
within  teleoperated  and  true  telepresence  systems. 

Fitts  (1954)  established  a  means  of  quantifying  human  performance  in 
terms  of  information  processing  capacity  of  the  neuromotor  system.  He 
developed  a  relationship  between  the  speed  and  the  accuracy  of  human  motor 
performance  and  demonstrated  that  the  speed  of  a  task  of  requiring  a 
particular  accuracy  is  bounded  by  the  capacity  of  the  neuromuscular  system 
control  movements.  Fitts  argued  that  if  manual  control  was  limited  by  the 
information  processing  rate  of  the  peripheral  and  central  nervous  system, 
movement  times  would  be  limited  by  the  information  processing  demands  of 
the  task.  This  concept  is  apparent  if  we  think  of  a  task  requiring  great 
accuracy  such  as  threading  a  needle.  Why  do  we  perform  a  threading  task 
with  infuriating  sluggishness?  Such  a  task  is  limited  by  human  information 
processing  capacity  (i.e.,  how  fast  we  can  perceive  the  environment,  actuate 
our  limbs,  and  adjust  for  error).  A  task  such  as  threading  a  needle  requires 
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many  fine  adjustments  and  is  thus  limited  by  how  quickly  perceptions  and 
adjustments  can  be  performed  (i.e.,  limited  by  human  bandwidth). 


To  measure  human  information  processing  capacity  as  an  indication  of 
performance  within  a  telepresence  environment,  subjects  were  tested  on  a 
peg  insertion  task.  Like  threading  a  needle,  the  difficulty  of  peg  insertion  is  a 
function  of  tolerance,  the  difference  between  the  peg  diameter  and  the  hole 
diameter.  The  tighter  the  fit  of  the  peg  in  the  hole,  the  more  accuracy 
required  of  the  operator,  and  thus  the  slower  the  maximum  performance 
speed.  By  requiring  subjects  to  perform  a  standardized  peg  insertion  task  as 
fast  as  they  can,  completion  times  could  be  measured  and  compared  to  task 
difficulty  to  yield  information  processing  capacity  for  that  task.  By  doing 
such  an  analysis  upon  subjects  with  and  without  the  implementation  of 
various  virtual  fixtures,  variations  in  task  completion  times  indicate 
performance  changes  resulting  from  fixture  use.  Thus,  by  using  Fitts'  Law 
analysis  of  a  standardized  peg  insertion  task,  the  effectiveness  of  various 
virtual  fixtures  as  perceptual  aids  could  be  quantified. 


Figure  1 :  Measurements  used  to  define  task  difficulty 
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Through  an  extensive  review  of  human  psychomotor,  perceptual,  and 
cognitive  test  batteries,  the  Naval  Oceans  Systems  Center  developed  a  peg 
insertion  performance  task  specifically  representative  of  teleoperator 
manipulative  activities  [16,  24],  The  test  battery  requires  subjects  to  move  pegs 
of  various  diameters  between  holes  of  varied  spacing.  Movement  times  for  peg 
motions  are  recorded  and  correlated  with  task  difficulty.  As  defined  by  Fitts 
[7],  the  binary  Index  of  Difficulty,  ID,  for  the  one  dimensional  peg  transfer 
task  can  be  computed  as: 


ID  =  -log2 


W 

2A 


(bits/response) 


where  A  is  the  amplitude  of  the  motion  and  W  is  the  peg  tolerance  defined  as 
the  difference  between  the  hole  diameter  and  the  peg  diameter.  These 
quantities  are  shown  in  Figure  1. 

Fitts'  Law  [7]  relates  task  completion  time  to  task  ID  by  defining  the 
movement  time,  mt,  as  follows: 


mt  =  k|  ID  +  k2  (sec) 

where  kj  and  k2  are  characteristic  constants  of  the  individual  operator  and 

represent  the  slope  and  intercept  of  the  Fitts'  Law  curve.  The  reciprocal  of  the 
slope  of  the  Fitts'  Law  curve  (l/kj)  has  units  (bits/sec)  (which  is  identical  to 

units  of  capacity  for  an  information  channel)  and  is  thus  an  accepted  measure 
of  human  information  processing  capacity  [17].  Because  the  slope  is  a  coarse 
measure  and  is  susceptible  to  distortions  resulting  from  changes  in  strategy 
between  tasks  of  different  difficulty,  a  more  robust  measure  of  processing 
capacity  was  also  computed,  called  the  Binary  Index  of  Performance.  This 
value,  abbreviated  as  Ip,  describes  the  information  processing  capacity 
required  of  the  operator  to  perform  a  task  of  a  particular  difficulty  [7].  Ip  also 
has  units  (bits/sec)  and  is  defined  as  follows: 

T  J-  ,  ^ 

Ip  =  log2  2 A  (bits/sec) 
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(or) 


ID 

Ip  =  ^  (bits/sec) 

^  m  t 

where  mt  is  the  movement  time  required  to  complete  the  task.  Index  of 

Performance  Ip  is  an  accepted  measure  of  the  information  processing  capacity 

of  the  teleoperator  [19]. 

Possible  Performance  Advantages  of  Virtual  Fixtures 

Fitts  [7]  demonstrated  that  performance  in  manual  tasks  is  limited  by  the 
information  processing  capacity  of  the  central  and  peripheral  nervous 
systems.  Virtual  fixtures  could  enhance  performance  by  either  reducing  the 
information  processing  demands  of  a  given  task  or  by  increasing  the 
information  processing  capacity  of  the  operator.  Since  the  fixtures  are 
perceptual  overlays  and  do  not  exist  in  the  workspace,  it  would  be  incorrect  to 
suggest  that  fixtures  could  in  any  way  modify  the  given  task.  If  fixtures 
cannot  alter  the  task,  how  can  they  act  to  reduce  the  processing  demands  of 
the  task?  Although  the  task  itself  remains  the  same,  virtual  fixtures  can 
modify  how  the  workspace  is  perceived,  alter  how  the  task  is  conceptualized, 
and  can  thus  be  designed  to  reduce  the  processing  demands  of  the  task.  For 
example,  by  implementing  haptic  fixtures  as  physical  guides  or  barriers,  the 
operator  might  have  fewer  alternative  movements  to  consider  in  the 

perception  of  the  workspace  than  really  exist  in  the  actual  task  workspace . 
Virtual  guides  or  barriers  simplify  the  perception  of  the  task  workspace, 
reduce  the  information  processing  demands  on  the  nervous  system,  and  thus 

increase  operator  performance.  Thompson  (1977)  demonstrated  a  similar 
effect  by  showing  that  completion  times  in  part  mating  tasks  could  be  reduced 

by  eliminating  degrees  of  constraint  [26].  Virtual  fixtures  do  not  eliminate 

constraints  required  by  a  task,  but  they  can  assist  the  operator  in  achieving 

some  of  the  required  constraints. 

It  is  also  hypothesized  that  the  additional  percepts  offered  by  virtual 
fixtures  provide  localizing  references  to  the  remote  workspace  (i.e., 
interaction  with  the  fixtures  gives  the  operator  a  better  sense  of  the  physical 
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relationship  between  kinesthetic  and  proprioceptive  output  and  workspace 
geometry).  Such  localization  enhances  the  illusion  of  presence,  provides  the 

operator  with  a  better  understanding  of  the  workspace,  and  reduces  the 

operator's  reliance  on  sensory  feedback.  This  localizing  effect  should  act  to 

reduce  the  overall  information  processing  demands  of  the  task  and  ultimately 
enhance  operator  performance. 

Finally,  virtual  fixtures  could  also  enhance  performance  by  providing 
alternative  information  pathways  by  introducing  feedback  from  additional 
sensory  modalities.  Since  the  nervous  system  is  parallel  in  architecture,  the 

introduction  of  alternative  pathways  could  act  to  increase  the  information 
processing  capacity  of  the  operator.  To  test  these  hypotheses,  the  following 
experiment  measured  operator  performance  with  and  without  the  use  of 
simple  virtual  fixtures.  The  fixtures  tested  were  chosen  to  offer  insight  into 
all  three  modes  of  operation  presented  above:  Simplification  of  the  perception 
of  the  task,  enhanced  localization  to  remote  workspace,  and  increased  capacity 
due  to  alternative  modes  of  sensory  feedback. 
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EXPERIMENTAL  HARDWARE 


Virtual  Fixture  Performance  Testbed 


To  implement  a  standardized  Fitts’  Law  performance  test  in  a 
telepresence  environment,  the  following  hardware  subsystems  were  used:  Peg 
Insertion  Task  Board,  MBA  Exoskeletal  Master/Merlin  Robot  Arm  Slave, 
Monocular  Vision  System,  and  Virtual  Fixture  Board. 


Figure  2:  Photograph  of  the  AF/Navy  Teleoperator  Performance 
Evaluation  Battery  Task  Board  used  for  implementing  peg 
insertion  Fitts’  Law  task.  Also  shown  is  the  Merlin  Robot  Arm 
used  as  the  slave  in  the  teleoperated  system. 


Task  Board:  (AF/Navy  Teleoperator  Performance  Evaluation  Battery) 


As  shown  in  Figure  2,  a  specialized  task  board  was  used  to  implement  the 
performance  task.  This  task  board  was  originally  developed  by  the  Naval 
Oceans  Systems  Center  to  implement  a  controlled  peg-in-hole  test  battery  for 
telemanipulation  systems  [16,  24].  Holes  on  the  board  are  arranged  to 
accommodate  a  wide  range  of  peg  sizes  and  movement  amplitudes.  This  study 
made  use  of  four  holes  (diameter  2.00  cm)  spaced  to  allow  horizontal  peg 
movements  of  4.00  cm  and  16.00  cm.  Three  pegs  (diameters  0.75  cm,  0.98  cm. 
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1.50  cm)  were  used  in  the  task  to  vary  the  peg  insertion  tolerance.  The  bottom 
of  each  hole  contains  a  microswitch  connected  to  a  PC  via  a  Digital  I/O 
controller  card.  This  PC  controls  and  monitors  the  peg  insertion  portion  of  the 
experiment.  The  computer  is  equipped  with  a  real-time  clock  for  measuring 
peg  movement  times  to  the  nearest  millisecond.  The  timer  is  started  when  the 
peg  is  extracted  from  the  start  hole  and  stopped  when  inserted  into  the  target 
hole. 

MBA  Exoskeletal  Master/Merlin  Robot  Arm  Slave 

As  shown  in  Figure  3,  the  MBA  Exoskeleton  Master  is  a  dual-arm,  full 
upper  body  exoskeleton  which  can  transduce  motion  in  seven  degrees  of 
freedom  for  each  arm  [12].  In  its  current  configuration,  the  device  is  used 
only  as  a  sensor;  no  force  information  is  reflected  back  to  the  user  through 
the  exoskeleton.  Thus,  peg  insertions  were  performed  based  only  on  visual 
and  auditory  feedback  from  the  workspace.  This  reduced  sensory  feedback 
environment  offered  a  good  testbed  for  the  prowess  of  virtual  fixtures. 


Figure  3:  Photograph  of  subject  wearing  MBA  kinesthetic 

exoskeleton  used  as  the  master  in  the  teleoperator  system. 
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In  this  experiment,  the  position  of  the  subject’s  right  hand  was  tracked 
and  fed  to  the  slave  robot  arm.  An  end-effector  was  constructed  for  the  slave 
robot  which  housed  the  peg  and  allowed  for  easy  interchange  of  the  various 
diameter  pegs.  Because  of  the  difficulty  of  the  teleoperated  peg  insertion  task 
without  force  feedback,  the  slave  robot  was  programmed  to  maintain  its  end- 
effector  orientation  perpendicular  to  the  board  at  all  times.  This  arrangement 
simplified  the  task  and  eliminated  the  possibility  of  subjects  torquing  the  peg 
in  the  hole.  To  further  simplify  the  task,  the  robot  was  constrained  to  move 
only  in  a  horizontal  plane  level  with  the  target  holes.  Thus,  the  task  only 

required  two-dimensional  positioning  of  the  peg  in  the  xy  plane  with  x 
defined  into  the  board,  and  y  defined  along  the  horizontal  row  of  target  holes 
as  shown  in  Figure  4.  It  should  be  noted  that  although  the  task  was 

constrained  to  planar  positioningit  was  sufficiently  difficult  to  require 
subjects  to  practice  for  two  45-minute  sessions  before  training  was  complete. 

The  task  was  designed  to  be  simple  in  concept  but  difficult  enough  that  the 
subjects  would  not  approach  the  upper  bound  of  the  slave  robot’s  ability  to 

perform  the  task. 


Figure  4:  Schematic  of  Task  Board  with  xy  plane  of  allowable  peg  motion. 

To  protect  the  task  board  and  the  robot,  the  peg  was  rigidly  constrained 
in  all  degrees  of  freedom  except  along  the  x  axis  pointing  into  the  task  board. 
Along  that  axis,  the  peg  floated  on  a  stiff  spring  so  that  strong  forward  impacts 
of  the  peg  with  the  board  would  not  jar  the  board  or  damage  the  robot. 
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Monocular  Vision  System 


A  previous  study  using  an  identical  task  board  performed  by  the  Naval 
Oceans  Systems  Center  has  shown  that  stereo  vision  feedback  offers  no 
performance  advantage  over  monocular  projections  used  in  this  peg  insertion 
task  [16,  24],  Thus,  a  simple  monocular  vision  system  was  implemented  to 

provide  visual  feedback  for  this  experiment.  The  system  was  designed  as  an 

inexpensive  means  of  creating  the  illusion  of  operator  presence  while 
providing  visual  feedback  of  the  workspace.  The  system  used  7X  power 

binocular  optics  focused  upon  a  distant  color  video  monitor  which  displayed 
the  output  of  a  single  camera  in  the  workspace  focused  on  the  task  board.  The 
monitor  was  placed  at  a  distance  from  the  optics  in  such  a  way  that  the 

magnification  of  the  video  image  created  the  illusion  that  the  task  board  was 

within  reach  of  the  operator’s  hands.  Robot  end-effector  motion  was  scaled  to 
match  operator  hand  motion  so  that  the  apparent  end-effector  position 
corresponded  to  the  user’s  kinesthetic  sense  of  arm  position.  Such  a 

correlation  between  kinesthetic  feedback  and  visual  feedback  of  robot  end- 
effector  position  greatly  enhanced  the  user’s  sense  of  presence  within  the 

workspace.  Testing  of  the  system  revealed  best  results  when  the  angle  of  the 
camera  incident  on  the  workspace  was  closely  matched  by  the  angle  that  the 

magnifying  optics  were  incident  upon  the  distant  video  monitor.  When  these 
angles  were  not  similar,  conflicting  perspective  cues  hindered  the  illusion  of 

presence.  Figure  5  includes  a  rough  schematic  of  the  vision  system  as  part  of 
the  overall  system  hardware. 

Virtual  Fixture  Board 

Rather  than  using  a  force  reflecting  exoskeleton  to  model  the  rigid 
impedance  surfaces  which  compose  the  virtual  fixtures,  it  was  thought  that 
the  preliminary  tests  of  virtual  fixtures  should  not  be  influenced  by  hardware 
limitations  of  force  reflecting  devices.  Thus,  a  Fixture  Board  was  designed 
which  allowed  real  rigid  surfaces  to  be  fabricated  from  acrylic  sheets.  The 
MBA  exoskeleton  would  interact  with  the  real  acrylic  surfaces  and  reflect  that 
information  to  the  user.  As  far  as  the  user  is  concerned,  the  haptic  perception 
of  these  surfaces  was  coming  from  the  exoskeleton  and  was  just  as  much 
"virtual"  information  as  if  it  was  truly  computer  generated.  The  benefit  of  this 
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approach  is  that  the  reflected  perception  of  these  surfaces  was  perfectly 
modeled.  The  surfaces  felt  crisp  and  real,  free  from  the  bandwidth  limitations 
of  most  force  reflecting  devices.  The  drawback  of  this  approach  was  that  more 
abstract  fixtures  were  impossible  to  generate  using  this  method. 

The  virtual  fixtures  were  constructed  out  of  acrylic  sheets  and 

positioned  upon  a  wooden  platform  in  front  of  the  user.  To  make  the  fixtures 
quickly  interchangeable,  they  were  constrained  and  positioned  on  the  wooden 
platform  by  locating  pins.  The  exoskeleton  was  fitted  with  a  teflon  cap  at  the 
end  of  the  hand  grip  which  was  used  as  the  contact  surface  between  the 

exoskeleton  and  the  fixture.  The  fixtures  were  treated  regularly  with  an  oil- 

based  lubricant  so  that  little  friction  was  perceivable  between  the  teflon  and 
the  acrylic  surfaces.  Although  frictional  surfaces  could  make  for  effective 
virtual  fixtures,  the  intent  of  this  study  was  to  look  at  simple  surfaces  modeled 
only  as  rigid  impedances  free  from  any  viscous  or  coulombic  damping. 


figure  5:  Complete  Telepresence  System  developed  to  implement 
testing  of  teleoperator  performance  in  a  standardized  peg 
in.  ertion  task  with  and  without  the  aid  of  virtual  fixtures. 


Although  the  haptic  fixtures  were  modeled  physically  rather  than 
computationally,  auditory  fixtures  were  tested  that  were  pure  computer 
simulations.  Simple  compliant  surfaces  were  modeled  in  which  the 
compression  of  the  surface  was  proportional  to  a  linear  change  in  pitch. 
Tones  were  generated  on  a  PC  and  fed  to  the  user  via  stereo  headphones.  These 
trials  served  as  a  testbed  for  alternate  modality  fixtures  as  well  as  a  proof  of 
concept  of  purely  computer-generated  fixtures. 

Overall  Test  Setup 

Having  described  each  of  the  major  components  of  this  test  setup,  it  is 
important  to  clearly  describe  the  system  as  a  whole  before  getting  into  the 
details  of  the  subject  testing.  As  shown  in  Figure  5,  the  system  is  divided  into 
two  physically  separate  parts:  the  remote  environment  and  the  operator 

space.  The  remote  environment  contains  the  task  board,  the  merlin  robot  arm, 
and  a  single  video  camera  pointed  at  task  board.  The  camera  is  positioned  so 
that  the  incident  perspective  is  similar  to  what  a  human  operator  would  see  if 
standing  directly  in  front  of  the  board  and  performing  the  peg  insertions  in 
person.  The  operator  space  contains  the  exoskeleton,  the  vision  system,  and 
the  virtual  fixture  board.  Once  inside  the  exoskeleton  and  vision  system,  the 
subject  is  presented  with  a  projection  of  the  image  from  the  camera  in  the 
remote  environment.  The  subject  is  given  the  illusion  that  the  task  board  is 
situated  directly  before  him,  within  reaching  distance  of  the  exoskeleton.  In 
reality,  the  task  board  is  on  the  opposite  side  of  the  laboratory,  behind  the 
subject  and  completely  out  of  view.  The  fixture  table  is  placed  directly  in  front 
of  the  subject  in  such  a  way  that  it  cannot  be  seen  when  looking  through  the 
vision  system,  but  feels  as  though  it  occupies  the  same  space  as  the  apparent 
image  of  the  task  board.  Thus,  virtual  fixtures  implemented  on  the  fixture 
board  feel  as  though  they  are  overlaid  on  top  of  their  perception  of  the  remote 
environment. 

The  remote  robot  arm  is  slaved  to  the  right  hand  position  of  the 
exoskeleton.  Thus,  when  the  subject  moves  his  right  hand  so  as  to  interact 
with  the  image  of  the  task  board,  the  end-effector  on  the  slave  arm  follows. 
Because  the  subject  cannot  see  his  own  hand  when  looking  into  the  vision 


16 


system  but  does  see  the  remote  robot  end-effector  in  the  position  where  he 
feels  his  hand  to  be,  a  sense  of  presence  within  the  remote  environment  is 
created.  The  subjects  also  wear  a  set  of  stereo  headphones  for  use  in 
implementing  auditory  perceptual  overlays.  Auditory  surfaces  and  fields  can 
be  modeled  on  the  control  computer  and  interacted  with  by  the  user.  The 
complete  system  provides  a  powerful  testbed  for  projecting  subjects  into  a 
remote  environment  and  overlaying  haptic  and  auditory  information  on  top  of 
the  reflected  percepts. 
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EXPERIMENT  DESIGN  &  PROCEDURE 


Experiment  Design 

The  goal  of  this  study  was  to  investigate  the  effect  that  the  presentation 
of  virtual  fixtures  has  upon  performance  in  a  standard  telemanipulation  task. 
Eight  simple  fixtures  were  developed  for  comparison  to  a  no-fixture  control 
case.  Six  of  the  test  fixtures  were  purely  haptic  sensations  while  two  fixtures 
introduced  both  haptic  and  auditory  sensory  information.  It  should  be  noted 
that  the  design  of  these  test  fixtures  was  not  motivated  by  finding  the  most 
effective  perceptual  aid  for  this  particular  peg  insertion  task.  This  particular 
set  of  test  fixtures  was  chosen  to  provide  general  insight  into  the  design  and 
use  of  virtual  fixtures  and  to  allow  the  evaluation  of  some  of  the  basic  building 
blocks  from  which  fixtures  can  be  made. 

When  designing  the  test  fixtures,  the  peg  insertion  task  was  not  thought 
of  as  a  single  motion,  but  rather  as  a  combination  of  two  phases:  ballistic 
motion  and  fine  positioning.  The  ballistic  phase  was  defined  as  the  time  from 
which  the  peg  is  removed  from  the  start  hole  until  it  makes  first  contact  with 
the  target  hole.  The  fine  positioning  phase  was  defined  to  begin  at  the  end  of 
the  ballistic  motion  and  continue  until  the  peg  is  properly  positioned  in  the 
hole  [17].  This  decomposition  of  the  peg  insertion  task  was  found  to  be  a  useful 
conceptual  guide  for  fixture  design.  Some  of  the  test  fixtures  were  targeted  at 
aiding  ballistic  motion,  some  were  targeted  at  enhancing  fine  motion,  and 
some  were  intended  to  help  both.  Figure  6  shows  a  schematic  representation 
of  each  of  the  eight  fixtures  tested.  These  fixtures  are  shown  overlaid  on  top  of 
the  task  board  as  they  are  perceived  by  the  subjects.  Although  these  fixtures 
are  represented  graphically  in  this  figure,  they  are  perceived  only  as  forces 
or  sounds  by  the  subjects.  Although  instilling  visual  qualities  to  fixtures  is  a 
viable  application  of  perceptual  overlays  and  an  important  topic  for  future 
investigation  into  virtual  fixtures,  this  study  was  restricted  only  to  haptic  and 
auditory  sensory  modalities. 
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Figure  6;  Virtual  fixtures  shown  projected  on  top  of  the  task 
board  as  perceived  by  the  subjects.  Also  shown  are  top  views, 
looking  down  from  above  the  task  board.  The  fixtures  are  shown 
graphically  here,  but  they  are  perceived  haptically  or  auditorily 
by  subjects. 
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As  shown  in  Figure  6,  the  eight  fixtures  tested  are  composed  of  simple 
combinations  of  planar  surfaces.  Fixture  1  is  simply  a  rigid  horizontal  surface 
oriented  like  a  table  top  in  the  workspace  and  positioned  so  that  contact  with 
the  surface  will  result  in  vertical  alignment  of  the  operator's  hand  with  the 
holes  in  the  peg  board.  Fixture  2  is  just  like  Fixture  1,  but  it  includes  a  second 
surface  which  is  parallel  to  the  plane  of  the  task  board  and  located  three 
inches  back  from  the  board.  When  using  this  fixture,  subjects  operate  in  the 
space  between  the  vertical  plane  and  the  task  board.  This  arrangement 
provides  some  limits  upon  the  operator’s  ballistic  motion.  Fixture  3  is  similar 
to  Fixture  2,  except  the  vertical  plane  is  angled  so  as  to  converge  with  the  task 
board  as  the  target  hole  is  approached.  This  arrangement  acts  like  a  funnel, 
confining  and  guiding  ballistic  motion  towards  the  target  hole. 


Figure  7:  Sample  trajectory  of  peg  from  start  hole  to  target  hole 
as  guided  by  a  virtual  fixture.  As  shown,  the  fixture  does  not 
define  the  motion  of  the  operator's  hand,  but  rather  defines  the 
boundaries  of  the  motion. 


Fixture  4  is  identical  to  Fixture  3,  except  a  third  plane  is  added  directly  in  front 
of  the  target  hole.  This  fixture  influences  the  ballistic  phase  even  further, 
stopping  hand  motion  when  close  to  the  target  hole.  Fixture  6  is  similar  to 
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Fixture  3,  except  a  second  funnel-like  fixture  is  added  in  front  of  the  target 
hole.  Thus,  ballistic  motion  is  guided  as  was  done  by  Fixture  3,  and  fine  motion 
is  guided  by  the  additional  surfaces.  Figure  7  shows  a  sample  trajectory 
superimposed  upon  Fixture  4,  making  the  fixture  implementation  clear.  This 

drawing  demonstrates  how  the  use  of  a  fixture  might  influence  the  trajectory 
of  the  peg.  As  shown  in  the  figure,  the  rigid  surfaces  do  not  define  the  motion 
of  the  peg,  but  rather  influence  the  operator's  trajectory  by  confining  the 

boundaries  of  peg  motion. 

Fixture  5  is  very  different  from  those  presented  thus  far  in  both  its 
geometry  and  implementation.  The  key  is  that  his  fixture  was  not  interacted 

with  by  the  subject's  right  hand  (the  hand  that  performed  the  peg  insertion 

task),  but  rather  was  designed  for  interaction  only  with  the  unused  left  hand. 
Fixture  5  is  a  rigid  impedance  plane  parallel  to  the  task  board  and  located 
approximately  0.5"  in  front  of  the  board.  The  subject  would  place  his  left  hand 
upon  the  planar  surface  while  performing  the  task  with  the  right  hand.  The 
intent  of  this  fixture  was  to  isolate  the  effect  of  localization  on  performance. 
It  was  hypothesized  that  all  fixtures  provided  some  localizing  information  to 
the  users  which  enhanced  their  understanding  of  the  geometry  of  the 
workspace  and  allowed  them  to  better  correlate  their  kinesthetic  sense  of  hand 
position  to  the  remote  site.  Because  interaction  with  other  fixtures  would 
guide  or  limit  the  operator’s  motion,  it  was  impossible  to  isolate  performance 
increases  due  to  the  localizing  effect  alone.  Thus,  Fixture  5  was  designed  to 
interact  only  with  the  unused  hand  and  therefore  only  affects  performance 
by  providing  localizing  cues. 

Fixture  7  and  Fixture  8  are  the  only  fixtures  which  implement  both 
haptic  and  auditory  information.  Fixture  7  is  identical  to  Fixture  1,  but  also 
introduces  a  texture-like  field  of  auditory  information.  The  information  is 
represented  as  a  series  of  surfaces  perpendicular  to  the  task  board  as  shown  in 
blaek  in  Figure  6.  Operator  interaction  with  these  surfaces  results  in  the 
production  of  an  audible  tone.  The  pitch  of  the  tone  increases  from  left  to 
right  across  the  task  board.  Fixture  8  is  identical  to  Fixture  6,  but  it  introduces 
a  single  compliant  surface  in  front  of  the  target  hole  as  shown  in  black  in 
Figure  6.  This  surface  is  modeled  as  a  compliant  surface  with  a  proportional 
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stiffness  such  that  interaction  with  this  surface  produces  a  tone  proportional 
to  compression. 


Experimental  Protocol 

A  series  of  tests  was  run  to  evaluate  subject  performance  using  each 
fixture  configuration.  Operator  performance  was  recorded  during  test  periods 
which  included  12  practice  and  36  timed  peg  insertion  trials  for  each  fixture 
studied.  A  single  trial  consisted  of  moving  a  peg  from  a  designated  start  hole  to 
a  designated  target  hole.  The  holes  were  referenced  by  numerals  (3,  4,  5  or  6) 
located  above  each  hole  as  shown  in  Figure  4.  Two  different  peg  motions  were 
studied  in  these  tests:  a  16  cm  motion  from  hole  3  to  hole  6  and  a  4  cm  motion 
from  hole  5  to  hole  6.  The  36  trial  period  was  divided  into  three  groups  of  12 
trials.  Each  of  these  groups  required  the  subject  to  perform  the  insertion  task 
using  a  different  peg  size.  After  the  completion  of  each  group  of  12  trials, 
subjects  were  instructed  to  rest  for  approximately  two  minutes  while  a  new 

peg  size  was  installed  in  the  robot  end-effector. 

The  use  of  two  motion  amplitudes  (4  cm  and  16  cm)  and  three  peg  sizes 

(0.75  cm,  0.98  cm,  1.50  cm  diameter  pegs)  allowed  for  the  testing  of  insertion 

trials  with  six  different  task  difficulties.  Table  1  shows  all  combinations  of  peg 
size  and  motion  amplitude  and  lists  the  Index  of  Difficulty  for  each  task  as 
dictated  by  Fitts'  Law  [7].  The  order  in  which  the  three  peg  sizes  were 

presented  to  each  subject  was  randomized  to  ensure  that  mental  and  physical 
fatigue  had  similar  effects  on  all  task  difficulties. 


Table  1;  Index  of  Difficulties  (ID)  shown  for  all  combinations  of 
peg  size  and  peg  motion  amplitudes  as  predicted  by  Fitts'  Law. 


Motion  Amplitude 


Diameter 

5  to  6  (4  cm) 

0.75  cm 

2.68  bits/response 

4.68  bits/response 

0.98  cm 

2.98  bits/response 

4.98  bits/response 

1.50  cm 

4.00  bits/response 
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Subjects  were  instructed  to  begin  each  test  period  with  a  3  to  6  peg 
insertion  trial.  Subjects  would  then  perform  a  5  to  6  peg  insertion  trial,  then  a 
3  to  6  peg  insertion  trial,  and  cycle  in  that  manner  throughout  the  36  trials. 

Subjects  were  allowed  to  proceed  through  the  test  period  at  their  own  pace.  To 
automate  the  testing  procedure  and  allow  subjects  to  proceed  through  the  test 
with  little  operator  intervention,  a  number  of  simple  beeps  were  used  to  guide 
subject  activity.  When  a  subject  inserted  a  peg  into  the  correct  starting  hole, 

the  task  board  control  computer  would  emit  an  audible  tone  to  signal  that  the 
peg  was  properly  positioned.  The  subject  would  then  keep  the  peg  in  the  start 
hole  for  two  seconds  until  the  control  computer  produced  a  second  tone  which 

signaled  that  the  task  could  now  be  performed  at  will.  The  reason  for  this  two 
second  waiting  period  was  to  ensure  that  the  subject  maintained  a  steady 

contact  with  the  microswitch  at  the  base  of  the  start  hole.  The  subjects  were 
not  required  to  perform  the  insertion  task  as  soon  as  the  second  tone  was 

heard;  this  second  tone  simply  meant  "you  are  free  to  go  whenever  ready. 

Thus,  the  subjects  could  mentally  prepare  themselves  for  the  particular 

insertion  task  and  begin  at  will.  When  ready,  the  subject  would  remove  the 
peg  from  the  start  hole  and  insert  it  into  the  target  hole  as  fast  as  possible. 
Upon  insertion  into  the  target  hole,  the  control  computer  would  emit  a  tone  to 
signal  that  the  trial  had  been  successfully  completed.  The  subject  was  then 

free  to  proceed  to  the  next  insertion  trial  at  will.  At  the  end  of  each  trial  the 
control  computer  recorded  the  movement  time  along  with  the  peg  size  and 
motion  amplitude.  After  the  completion  of  each  group  of  12  trials,  the  control 
computer  would  emit  a  long-duration  tone  which  signaled  the  subject  to  rest 
for  two  minutes  while  the  operator  exchanged  peg  sizes.  Post  testing 
interviews  revealed  that  all  subjects  were  comfortable  with  the  use  of  audible 
tones  to  automate  the  testing  procedure. 

Subjects  were  tested  over  9  experimental  sessions,  each  lasting  45  to  60 
minutes.  To  minimize  the  effects  of  fatigue  and  boredom,  no  subject  completed 
more  than  one  experimental  session  during  a  single  day  of  testing.  Each  of  the 
first  two  experimental  sessions  included  two  test  periods  of  36  trials.  The  first 
period  of  each  session  was  performed  with  no  fixture  and  the  second  was 
performed  with  Fixture  1.  These  initial  144  trials  were  treated  only  as  practice 
during  which  the  subjects  familiarized  themselves  with  the  use  of  the 
exoskeleton,  merlin  robot  arm,  vision  system,  and  fixture  table.  It  was  found 
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that  by  the  end  of  the  second  practice  session,  all  subjects  had  sufficiently 

learned  the  task  that  variability  in  movement  times  for  trials  of  the  same 

difficulty  had  fallen  below  20%  for  every  subject,  with  a  mean  variability  of 
14%  for  all  subjects.  Once  learning  had  stabilized,  subjects  were  sequentially 

tested  using  each  of  the  test  fixtures.  One  or  two  new  fixtures  were  tested 

during  each  daily  session  in  addition  to  a  baseline  fixture  which  was  tested 

during  every  session.  Fixture  1  was  the  baseline  fixture  and  was  used  to  track 

performance  increases  resulting  from  day-to-day  learning  so  that  any  such 

effects  could  be  compensated.  Table  2  lists  all  of  the  daily  sessions  along  with 

the  fixtures  tested  during  that  session. 


Table  2:  Testing  Schedule:  All  seven  tests  shown  along 

with  corresponding  daily  sessions  and  fixtures  tested  on 
that  day. 


TEST 

SESSION 

FIXTURES  STUDIED 

Practice 

day  1 

No  Fixture  /  Fixture  1 

Practice 

dav  2 

No  Fixture  /  Fixture  1 

Test  I 

day  3 

No  Fixture  /  Fixture  1 

Test  II 

Fixture  1  /  Fixture  2  /  Fixture  3 

Test  III 

Fixture  1  /  Fixture  4 

Test  IV 

dav  6 

Fixture  0  /  Fixture  5 

Test  V 

Fixture  1  /  Fixture  6 

Test  VI 

day  8 

Fixture  1  /  Fixture  8 

Test  VII 

Fixture  1  /  Fixture  7 

24 


RESULTS 


The  following  section  presents  all  results  of  fixture  performance 
comparisons  in  tabular  form.  The  basic  quantities  presented  are  the  recorded 
movement  times  and  the  coefficients  of  variation  of  movement  times  for  trials 
of  the  same  difficulty.  The  coefficient  of  variation  is  a  measure  of  subject 
consistency  at  a  particular  task  and  is  an  indication  of  the  repeatability  of  the 
results.  Also  computed  is  the  slope  of  a  linear  regression  line  relating 

movement  time  to  index  of  difficulty.  The  inverse  of  the  slope  has  units 

(bits/sec)  and  is  a  measure  of  the  operator  information  processing  capacity. 
Because  the  slope  is  susceptible  to  distortions  resulting  from  changes  in 
strategy  between  tasks  of  different  difficulty,  a  more  robust  measure  of 
processing  capacity  was  also  computed  called  the  Binary  Index  of 
Performance.  This  value,  abbreviated  as  Ip,  describes  the  information 
processing  capacity  required  of  the  operator  to  perform  a  task  [7]. 

Tables  3  through  5  list  the  mean  movement  time  results  for  each  fixture 

tested.  Tables  6  through  8  list  the  computed  Index  of  Performance  (Ip)  for 

each  fixture  tested.  Table  9  lists  the  coefficients  of  variation,  slopes,  and  the 
inverse  of  the  slopes  for  each  fixture  tested.  Figure  8  graphically  compares 
the  performance,  showing  the  mean  movement  time  versus  index  of  difficulty 
curves  for  all  test  fixtures. 


TABLE  3:  Mean  Movement  Times  for  each  Index  of  Difficulty  are  shown  for  8 
subjects  using  no  fixture  as  well  as  using  Fixtures  1,  2,  3,  4  and  6.  Also  computed 
is  the  mean  movement  time  across  all  trials  (across  task  difficulty). 


INDEX  OF  DIFFICULTY 

FIXTURE 

2.68 

2.98 

4.00 

4.68 

4.98 

6.00 

mean 

No  Fixture 

1123ms 

1280ms 

1353ms 

1946ms 

1866ms 

2162ms 

1622ms 

Fixture 

1 

957ms 

977ms 

1118ms 

1595ms 

1656ms 

1758ms 

1344ms 

Fixture 

2 

842ras 

883ms 

1030ms 

1435ms 

1527ms 

1631ms 

1225ms 

Fixture 

3 

682ms 

704ms 

795ms 

1293ms 

1350ms 

1469ms 

1049ms 

Fixture 

4 

692ms 

713ms 

815ms 

1376ras 

1471ms 

1543ms 

1102ms 

Fixture 

6 

667ms 

722ms 

835ms 

1301ms 

1385ms 

1495ms 

1068ms 
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TABLE  4:  Mean  Movement  times  for  each  Index  of  Difficulty  are  shown  for  6 
subjects  using  no  fixture  as  well  as  using  the  localizing  Fixture  5  with  the 
unused  hand.  Also  computed  is  the  mean  movement  time  across  all  trials 
(across  task  difficulty). 


FIXTURE 

2.68 

IND 

2.98 

lEX  OF  1 

4.00 

3IFFICU] 

4.68 

LTY 

4.98 

6.00 

mean 

No  Fixture 

833ms 

92  7ms 

991ms 

1617ms 

1600ms 

1666ms 

1272ms 

Fixture  5 

656ms 

662ms 

807ms 

1448ms 

1396ms 

1497ms 

1078ms 

TABLE  5:  Mean  Movement  Times  for  each  Index  of  Difficulty  are  shown  for  6 
subjects  using  no  fixture,  using  purely  haptic  Fixtures  1  and  6,  as  well  as  using 
auditory /haptic  Fixtures  7  and  8.  Also  shown  is  the  mean  movement  time 
across  all  trials.  It  should  be  noted  that  Fixtures  1  and  7  are  identical  except  for 
the  addition  of  auditory  information  in  7.  The  same  is  true  for  Fixture  6  and 
Fixture  8. 


FIXTURE 

2.68 

IND 

2.98 

EX  OF  E 

4.00 

)IFFICU1 

4.68 

.TY 

4.98 

6.00 

mean 

No  Fixture 

1073ms 

1177ms 

1259ms 

1845ms 

1832ms 

2058ms 

1541ms 

Fixture  1 

840ms 

910ras 

969ms 

1539ms 

1569ras 

1674ms 

1250ms 

Fixture  7 

687ms 

727ms 

718ms 

1310ms 

1325ms 

13  69ms 

1023ms 

Fixture  6 

641ms 

691ms 

811ms 

1266ms 

1361ms 

1461ms 

1039ms 

Fixture  8 

614ms 

610ms 

701ms 

1136ms 

1154ms 

1270ms 

915ms 

TABLE  6:  Index  of  Performance,  Ip  =  (ID  /  mean  movement  time),  is  computed 
and  shown  for  8  subjects  using  no  fixture  as  well  as  using  Fixtures  1,  2,  3,  4  and 
6.  Ip  has  units  (bits/sec)  and  is  an  accepted  measure  of  human  information 
processing  capacity. 


FIXTURE 

2.68 

INDE] 

2.98 

SC  OF  DII 

4.00 

TTCULT' 

4.68 

Y 

4.98 

6.00 

mean  Ip 

No  Fixture 

2.34 

2.33 

2.96 

2.40 

2.67 

2.77 

2.58 

Fixture  1 

2.80 

2 . 99 

3.58 

2.93 

3.01 

3.41 

3.12 

Fixture  2 

3.18 

3.37 

3.88 

3.26 

3.26 

3.68 

3.44 

Fixture  3 

3.93 

4.23 

5.03 

3.62 

3.69 

4.08 

4.10 

Fixture  4 

3.87 

4.18 

4 . 91 

3.40 

3.39 

3.89 

3 . 94 

Fixture  6 

4.02 

4  . 13 

4.79 

3.60 

3.60 

4.01 

4.02 

TABLE  7:  Index  of  Performance,  Ip  =  (ID  /  mean  movement  time),  is  eomputed 
and  shown  for  6  subjects  using  no  fixture  as  well  as  using  the  localizing  Fixture 
5  with  the  unused  hand.  Ip  has  units  (bits/sec)  and  is  an  accepted  measure  of 
human  information  processing  capacity. 


HXTURE 

2.68 

INDE 

2.98 

X  OF  D1 

4.00 

FFICULH 

4.68 

PY 

4.98 

6.00 

mean  Ip 

No  Fixture 

3.22 

3.21 

4.03 

2.89 

3.11 

3.60 

3.60 

Fixture  5 

4.09 

4.50 

4.96 

3.23 

3.58 

4.01 

4.01 

TABLE  8:  Index  of  Performance,  Ip  =  (ID  /  mean  movement  time)  ,  is  computed 
and  shown  for  6  subjects  using  no  fixture,  using  purely  haptic  Fixtures  1  and 
6,  as  well  as  using  auditory /haptic  Fixtures  7  and  8.  Ip  has  units  (bits/sec)  and 
is  an  accepted  measure  of  human  information  processing  capacity.  It  should 
be  noted  that  Fixtures  1  and  7  are  identical  except  for  the  addition  of  auditory 
information  in  7.  The  same  is  true  for  Fixture  6  and  Fixture  8. 


HXTURE 

2.68 

IND 

2.98 

EX  OF  I 

4.00 

)IFFICUI 

4.68 

..TY 

4.98 

6.00 

mean  Ip 

No  Fixture 

2.50 

2.67 

3.18 

2.54 

2.72 

2.92 

2.76 

Fixture  1 

3.27 

4.12 

3.04 

3.17 

3.58 

3.40 

Fixture  7 

3.90 

4.01 

5.57 

3.57 

3.76 

4.38 

4.20 

Fixture  6 

4.18 

4.31 

4.93 

3.70 

3.66 

4.11 

4.15 

Fixture  8 

4.36 

4.88 

5.71 

4.12 

4.32 

4.72 

4.69 

Table  9;  Mean  Coefficient  of  Variation  for  all  subjects  performing  tasks  of  the 
same  difficulty  with  each  fixture  is  shown.  Also  shown  is  the  slope  and 
standard  error  of  a  linear  regression  analysis  of  movement  time  against  index 
of  difficulty.  The  reciprocal  of  the  slope  is  also  shown  and  has  units  (bits/sec). 
This  value  is  an  accepted  measure  of  information  processing  capacity. 


FIXTURE 

MEAN 

c.o.v. 

SLOPE 

[ms/bit] 

Std  Err 
of  Coef 

1 /SLOPE 
[bits  /  sec] 

No  Fixture 

13.7%  +2.5 

317 

43 

3.15 

Fixture 

1 

11.9%  ±2.0 

276 

39 

3.62 

Fixture 

2 

12.8%  ±3.2 

2  67 

37 

3.75 

Fixture 

3 

11.1%  ±2.7 

271 

47 

3.69 

Fixture 

4 

8.6%  ±1.9 

300 

56 

3.33 

Fixture 

5 

13.0%  ±1.3 

302 

62 

3.31 

Fixture 

6 

9.6%  ±2.8 

280 

44 

3.57 

Fixture 

7 

11.2%  ±1.9 

244 

61 

4.10 

Fixture 

8 

9.2%  ±1.5 

228 

40 

4.39 
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Performance  Curves  for  Each  Virtual  Fixture  Tested 
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Figure  8:  Performance  using  each  fixture  shown  for  comparison 
as  best  line  fit  to  Fitts'  Law  relation  between  mt  and  ID 

In  order  to  quantify  performance  increase  due  to  fixture  use  in  a 

meaningful  way  that  is  resistant  to  variations  across  task  difficulty, 
percentage  changes  in  movement  times  and  Index  of  Performance  were 

computed  for  each  fixture  with  respect  to  the  no-fixture  case.  The  percentage 
decrease  in  movement  times  is  a  unitless  measure  of  the  effectiveness  of  a 

fixture  to  speed  operator  performance  in  a  task.  The  percentage  increase  in 
Index  of  Performance  is  a  unitless  measure  of  the  increase  in  the  information 
processing  capacity  of  the  operator  or  the  decrease  in  processing 
requirements  of  the  task  resulting  from  fixture  use.  Tables  10  and  11  list  the 
percentage  decrease  in  movement  times  corresponding  to  the  use  of  each 

fixture.  Tables  12  and  13  list  the  percentage  increase  in  processing  capacity  Ip 
associated  with  the  use  of  each  fixture. 
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TABLE  10;  Percent  Decrease  in  Movement  Time  for  each  fixture  with  respect 
to  the  no-fixture  case.  Values  are  computed  for  each  index  of  difficulty.  Also 
shown  is  the  mean  percentage  decrease  in  movement  time  for  each  fixture. 
This  mean  value  represents  the  effectiveness  of  a  fixture  to  enhance  operator 
performance. 


INDEX  OF  DIFFICULTY 

HXTURE 

2.68 

2.98 

4.00 

4.68 

4.98 

6.00 

mean 

Fixture 

1 

15.96% 

mm 

19.02% 

19.82% 

11.93% 

20.61% 

19.03% 

±4 . 9 

Fixture 

2 

28 . 60% 

36.71% 

27.11% 

30.23% 

19.98% 

28.00% 

28.44% 

+5.4 

Fixture 

3 

48 .86% 

58.06% 

51.96% 

40.32% 

32.09% 

38.17% 

44.91% 

+  9.1 

Fixture 

4 

47.49% 

56.90% 

49.63% 

34.32% 

23.67% 

33.41% 

40.90% 

±11.9 

Fixture 

5 

23.77% 

33.35% 

20.47% 

11.03% 

13.62% 

10.69% 

18.82% 

+  7 . 9 

Fixture 

6 

50.95% 

55.74% 

47.35% 

39.73% 

29.59% 

36.48% 

43.31% 

+  9.2 

Fixture 

7 

43.86% 

47.27% 

54.73% 

33.91% 

32.12% 

40.21% 

42.02% 

+8.4 

Fixture 

8 

54.42% 

63.46% 

56.94% 

47.57% 

45.41% 

47.36% 

52.52% 

+  6.5 

TABLE  11:  Percentage  Decreases  in  Movement  Times  when  using  Fixtures  7  and 
8  as  compared  to  Fixtures  1  and  6  respectively.  Since  Fixtures  7  and  8  are 
identical  to  1  and  6  in  all  ways  except  for  the  addition  of  auditory  information, 
the  percentage  decrease  in  movement  times  reflects  the  performance 
advantage  associated  with  overlaying  auditory  information. 


FIXTURE 

2.68 

INE 

2.98 

)EX  OF 
4.00 

DIFFK 

4.68 

:ULTY 

4.98 

6.00 

mean 

Fixture  7 

20.04% 

22.36% 

29.76% 

16.08% 

16.86% 

20.05% 

20.89%  ±4.9 

Fixture  8 

4.30% 

12.45% 

14.55% 

10.82% 

16.46% 

13.99% 

12.10%  ±4.2 

TABLE  12:  Percent  Increase  in  Processing  Capacity  Ip  for  each  fixture  with 
respect  to  the  no-fixture  case.  Values  are  computed  for  each  index  of 
difficulty.  Also  computed  is  the  mean  percentage  increase  in  processing 
capacity  associated  with  each  fixture. 


FIXTURE 

2.68 

INE 

2.98 

)EX  OF 
4.00 

DIFFK 

4.68 

:ULTY 

4.98 

6.00 

mean 

Fixture  1 

19.66% 

28.33% 

20.95% 

22.08% 

12.73% 

23 . 10% 

21.14%  ±5.0 

Fixture  2 

35.90% 

44 . 64% 

31.08% 

35.83% 

22 .10% 

32.85% 

33.73%  ±  7.4 

Fixture  3 

67 . 95% 

81.55% 

69.93% 

50.83% 

38.20% 

47.29% 

59.29%  15.3 

Fixture  4 

65.38% 

79.40% 

65.88% 

41.67% 

26.97% 

40.43% 

53.29%  ±18.9 

Fixture  5 

27.02% 

40.19% 

23.08% 

11.76% 

15.11% 

11.39% 

20.31%  ±10.8 

Fixture  6 

71.79% 

77.25% 

61.82% 

50.00% 

34.83% 

44.77% 

56.74%  ±15.3 

Fixture  7 

56.00% 

50.19% 

75.16% 

40.55% 

38.24% 

50.00% 

51.69%  ±13.2 

Fixture  8 

74 .40% 

82.77% 

79.56% 

62.20% 

58 . 82% 

61.64% 

69.90%  ±  9.5 

TABLE  13;  Percentage  Increases  in  Processing  Capacity  Ip  when  using 
Fixtures  7  and  8  as  compared  to  Fixtures  1  and  6  respectively.  Since  Fixtures  7 
and  8  are  identical  to  1  and  6  in  all  ways  except  for  the  addition  of  auditory 
information,  the  percentage  increase  in  Ip  reflects  the  performance 
advantage  associated  with  overlaying  auditory  information. 


INDEX  OF  DIFFICULTY 

FIXTURE 

2.68 

2.98 

4.00 

4.68 

4.98 

6.00 

mean 

Fixture  1 

22.26% 

22.63% 

35.19% 

17.43% 

18.61% 

22.35% 

23.08%  ±6.3 

Fixture  8 

4.31% 

13.23% 

15.82% 

11.35% 

18.03% 

14 .84% 

12 . 93%  +4 . 7 
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DISCUSSION 

Analysis  of  Each  Test  Fixture 


The  following  section  sequentially  addresses  the  change  in 
performance  associated  with  each  fixture  and  discusses  the  implications.  The 
objective  of  this  analysis  is  to  gain  general  insight  into  the  fundamental 
properties  of  virtual  fixtures  in  order  to  facilitate  the  design  and 
implementation  of  an  interactive  perceptual  workstation. 

Before  comparing  the  performance  results  for  task  completion  with 
each  fixture  configuration,  the  issue  of  repeatability  and  reliability  of  the 

results  should  be  addressed.  As  shown  in  Table  9,  the  coefficients  of  variation 

for  task  completion  is  under  14%  for  all  fixture  configurations,  with  a  mean 

variability  of  10%.  This  means  that  for  any  given  task  of  a  particular 
difficulty  performed  by  any  given  subject,  the  variation  in  performance  for 
all  such  trials  was  on  average  10%.  Such  consistency  for  human  performance 
is  surprisingly  good  and  suggests  that  the  results  of  these  tests  are  highly 
reliable. 

Virtual  Fixture  1 

As  shown  in  Figure  6,  virtual  Fixture  1  is  a  rigid  impedance  plane 
oriented  perpendicularly  to  the  plane  of  the  task  board.  When  interacting 

with  this  fixture,  the  subject’s  hand  motion  is  restrained  to  move  only  in  the 
plane  of  the  peg  holes.  Thus,  this  fixture  reduces  the  Cartesian  degrees  of 
freedom  on  hand  motion  from  three  to  two.  An  interesting  point  about  this 
fixture  is  that  the  end-effector  of  the  slave  robot  is  restricted  by  software  to 
planar  two  degree  of  freedom  motion  regardless  of  the  operator's  commands. 
Thus,  the  degree  of  freedom  which  is  eliminated  by  the  use  of  Fixture  1  has  no 
effect  on  the  position  of  the  slave  robot  arm.  Since  this  degree  of  freedom 
plays  no  part  in  performing  the  task,  it  would  seem  that  Fixture  1  would  have 
little  effect  on  the  performance  of  the  operator. 

As  shown  in  Tables  3  and  10,  the  mean  movement  times  for  peg 
insertions  performed  with  the  aid  of  virtual  Fixture  1  was  19%  faster  than  with 
the  no-fixture  case.  Comparing  Indexes  of  Performance  (Ip)  as  shown  in 


Tables  6  and  12,  the  human  information  processing  capacity  when  using 

virtual  Fixture  1  was  21%  greater  than  with  no  fixture  at  all.  This  comparison 
suggests  that  a  virtual  fixture  can  enhance  performance  by  simplifying  the 
perception  of  the  task  even  though  the  task  itself  is  unchanged.  In  both  cases, 
the  peg  insertion  task  was  a  2  degree  of  freedom  operation.  Without  the 

fixture,  the  subjects  were  free  to  move  in  all  three  Cartesian  directions  even 
though  they  were  well  aware  that  the  slave  robot’s  position  was  locked  in  the  y 
direction.  With  the  fixture,  the  subjects  were  guided  to  move  only  in  the 
plane.  This  implies  that  even  after  the  144  trials  of  practice,  the  subjects  were 
unable  to  completely  ignore  the  irrelevant  degree  of  freedom  and  were 

wasting  processing  capacity  on  it.  This  result  suggests,  first,  that  use  of  the 

single  rigid  impedance  plane  as  a  virtual  fixture  was  effective  in  significantly 
increasing  operator  performance.  Second,  this  result  suggests  that  when 

developing  a  presence  system  which  interfaces  a  human  limb  to  a  robot  arm 
with  fewer  degrees  of  freedom,  the  unused  degrees  of  freedom  may  degrade 
performance  and  thus  should  be  restrained  whenever  possible. 

Think  back  to  the  example  given  earlier  which  proposed  the  use  of  a 
virtual  fixture  similar  to  Fixture  1  to  enhance  performance  in  a  telepresence 
system  for  surgery.  A  single  rigid  impedance  plane  was  described  which  could 

be  located  at  some  depth  below  the  tissue  of  a  patient  such  that  interaction  of 
the  surgeon's  scalpel  with  this  rigid  surface  could  assure  that  the  incision 
only  reached  a  particular  depth,  thus  protecting  vital  organs.  The  results  of 
testing  with  Fixture  1  suggest  that  not  only  would  such  a  fixture  limit  motion  to 
the  plane  and  thus  maintain  a  constant  incision  depth,  it  would  also  enhance 
operator  ability  to  perform  in  free  plane  and  increase  information  processing 

capacity.  Thus,  not  only  would  the  surgeon  be  safe  from  encountering  vital 
organs  located  beneath  the  protective  surface,  the  surgeon  would  have 
enhanced  dexterity  above  the  surface. 
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Virtual  Fixture  2 


As  shown  in  Figure  6,  virtual  Fixture  2  is  the  same  as  Fixture  1  except  a 
second  rigid  planar  impedance  surface  is  added  parallel  to  the  surface  of  the 
task  board.  Like  Fixture  1,  Fixture  2  restricts  operator  hand  motion  to  the 

horizontal  plane.  The  added  surface  also  restricts  how  far  back  from  the  board 
the  operator  can  draw  the  peg.  As  shown  in  Tables  3  and  10,  the  mean 
movement  times  for  peg  insertions  performed  with  the  aid  of  virtual  Fixture  2 
were  28%  faster  than  with  the  no-fixture  case.  Comparing  Indexes  of 

Performance  (Ip)  as  shown  in  Tables  6  and  12,  the  human  information 
processing  capacity  when  using  virtual  Fixture  2  was  34%  greater  than  with 
no  fixture  at  all.  This  comparison  clearly  shows  that  the  use  of  Fixture  2 
significantly  enhanced  performance  over  the  no-fixture  case  and  that  use  of 

the  additional  surface  in  Fixture  2  as  compared  to  Fixture  1  made  a  significant 
improvement.  The  question  thus  remains,  why  should  the  additional  surface 
in  Fixture  2  increase  performance?  It  is  believed  that  the  additional  surface 
serves  a  number  of  purposes  by  restricting  how  far  back  the  peg  can  be 

drawn  from  the  board:  First,  this  surface  guides  operator  ballistic  motion, 
preventing  unnecessarily  wide  trajectories.  This  should  reduce  the 
information  processing  required  of  the  task  because  the  distance  from  the  task 
board  is  no  longer  a  parameter  the  operator  needs  to  be  constantly  concerned 
with.  Secondly,  this  surface  acts  as  a  kinesthetic/proprioceptive  localizing 
agent,  giving  the  user  a  better  sense  of  hand  position  in  the  remote  workspace 
by  providing  a  haptic  indication  of  proximity  from  the  task  board.  Thirdly,  by 
providing  depth  information  haptically,  the  fixture  reduces  the  demand  on  the 
visual  system  to  gage  depth  and  frees  up  that  modality  for  other  uses,  like 
tracking  the  target  hole. 

Virtual  Fixture  3 

As  shown  in  Figure  6,  virtual  Fixture  3  is  the  same  as  Fixture  2  except 
the  second  rigid  planar  impedance  surface  is  not  parallel  to  the  surface  of  the 
task  board,  but  rather  is  diagonal.  Like  Fixture  1,  Fixture  3  restricts  operator 
hand  motion  to  the  plane.  Like  Fixture  2,  the  additional  surface  restricts  how 
far  back  from  the  board  the  operator  can  draw  the  peg.  The  unique  thing 
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about  Fixture  3  is  that  it  does  not  uniformly  restrict  how  far  back  the  peg  can 
be  drawn,  but  rather  converges  hand  motion  toward  the  board  as  the  target 
hole  is  approached.  Although  this  fixture  is  composed  of  2  intersecting  planes, 
it  can  be  thought  of  as  a  cone  projected  into  the  two  dimensional  workspace 
which  funnels  ballistic  motion  towards  the  target  hole.  As  shown  in  Tables  3 
and  10,  the  mean  movement  times  for  peg  insertions  performed  with  the  aid  of 
virtual  Fixture  3  were  45%  faster  than  with  the  no-fixture  case.  Comparing 
Indexes  of  Performance  (Ip)  as  shown  in  Tables  6  and  12,  the  human 
information  processing  capacity  when  using  virtual  Fixture  3  was  59%  greater 
than  with  no  fixture  at  all.  Like  Fixture  2,  this  fixture  provides  haptic  depth 
cues  and  kinesthetic/proprioceptive  localization  to  the  remote  workspace.  The 
primary  advantage  of  this  fixture  is  that  it  guides  ballistic  motion  more 
efficiently,  funnelling  the  gross  motion  towards  the  target  hole.  It  is  believed 
that  this  perceptual  aid  in  target  convergence  reduces  the  demand  upon 
kinesthetic  and  visual  feedback  and  thus  reduces  the  information  processing 
required  for  the  task. 

Virtual  Fixture  4 

As  shown  in  Figure  6,  virtual  Fixture  4  is  the  same  as  Fixture  3  except 
for  the  addition  of  a  third  rigid  impedance  surface  which  crosses  the  diagonal 
surface.  This  additional  surface  was  placed  such  that  the  operator  would 
contact  this  surface  when  the  peg  was  directly  in  front  of  the  target  hole. 
Whereas  Fixture  3  aided  ballistic  motion  by  converging  hand  position  towards 
the  target  hole.  Fixture  4  was  intended  to  provide  further  trajectory  shaping 
by  halting  hand  motion  when  the  peg  was  aligned  with  the  hole.  As  shown  in 
Tables  3  and  10,  the  mean  movement  times  for  peg  insertions  performed  with 
the  aid  of  virtual  Fixture  4  were  41%  faster  than  with  the  no-fixture  case. 
Comparing  Indexes  of  Performance  (Ip)  as  shown  in  Tables  6  and  12,  the 
human  information  processing  capacity  when  using  virtual  Fixture  4  was  53% 
greater  than  with  no  fixture  at  all.  The  results  for  Fixture  4  are  not 
significantly  different  from  those  for  the  Fixture  3  case,  showing  that  the 
addition  of  the  third  surface  did  not  enhance  performance  in  these  trials.  It  is 
thus  concluded  that  the  additional  surface  added  no  important  information  for 
task  completion.  An  interesting  side  note  is  that  in  a  time  delay  study 
currently  being  completed.  Fixture  4  was  found  to  provide  significant 
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improvement  over  Fixture  3  when  communication  delays  were  introduced  into 
the  system.  This  makes  sense  because  additional  localizing  references  become 
more  important  when  time  delays  distort  the  relation  between  kinesthetic 
sense  of  position  and  visual  feedback  of  position.  These  results  will  be 
presented  formally  in  a  companion  paper  addressing  time  delay  issues. 

Virtual  Fixture  5 

As  shown  in  Figure  6,  virtual  Fixture  5  is  composed  of  a  single  rigid 
impedance  plane  which  is  parallel  to  the  task  board  surface  and  is  located 
approximately  0.5"  in  front  of  the  board.  Whereas  all  fixtures  thus  far 
described  were  designed  for  interaction  with  the  operator's  right  hand  (the 
hand  that  manipulates  the  peg).  Fixture  5  is  designed  for  interaction  only  with 
the  operator's  unused  hand.  Subjects  place  the  palm  of  their  left  hand  upon 
the  surface  while  they  perform  the  task  with  their  right  hand.  The  purpose  of 
testing  this  fixture  was  to  isolate  the  effect  of  localization  upon  performance 
because  this  fixture  can  only  influence  performance  through  pure 
localization.  The  idea  for  this  fixture  was  developed  while  observing  operators 
perform  the  peg  insertion  task  in  person  (not  telepresent).  All  subjects  rested 
their  unused  left  hand  upon  the  task  board  while  performing  the  task  with 
their  right  hand.  This  suggested  that  the  operators  gathered  useful  localizing 
information  from  the  unused  hand.  As  shown  in  Tables  3  and  10,  the  mean 
movement  times  for  peg  insertions  performed  with  the  aid  of  virtual  Fixture  5 
were  19%  faster  than  with  the  no*fixture  case.  Comparing  Indexes  of 
Performance  (Ip)  ns  shown  in  Tables  6  and  12,  the  human  information 
processing  capacity  when  using  virtual  Fixture  5  was  20%  greater  than  with 
no  fixture  at  all.  These  results  suggest  that  kinesthetic/proprioceptive 
localization  to  the  remote  site  plays  an  important  part  in  the  effectiveness  of 
virtual  fixtures  to  enhance  performance.  These  results  further  suggest  the 
importance  of  telepresence  systems  which  allow  for  bilateral  (dual  arm) 
interaction  with  the  workspace  even  if  for  tasks  which  only  require  one  hand. 


35 


Virtual  Fixture  6 


As  shown  in  Figure  6,  virtual  Fixture  6  is  identical  to  Fixture  3  with  the 
addition  of  two  angled  surfaces  which  guide  hand  motion  directly  into  the 
target  hole.  While  Fixture  3  guides  ballistic  motion  by  converging  near  the 
target  hole,  Fixture  6  also  guides  fine  motion  by  converging  hand  position 
directly  into  the  center  of  the  target  hole.  Although  this  fixture  is  composed 

of  4  intersecting  planes,  it  can  be  thought  of  as  two  cones  projected  into  the 

two  dimensional  workspace.  One  cone  funnels  ballistic  motion  across  the 
length  of  the  board;  the  other  cone  funnels  fine  motion  into  the  target  hole. 
As  shown  in  Tables  3  and  10,  the  mean  movement  times  for  peg  insertions 

performed  with  the  aid  of  virtual  Fixture  6  were  43%  faster  than  with  the  no¬ 
fixture  case.  Comparing  Indexes  of  Performance  (Ip)  as  shown  in  Tables  6  and 
12,  the  human  information  processing  capacity  when  using  virtual  Fixture  6 

was  57%  greater  than  with  no  fixture  at  all.  The  results  for  Fixture  6  are  not 
statistically  different  from  those  for  the  Fixture  3  case,  thus  showing  that  the 
addition  of  the  fine  positioning  surfaces  did  not  enhance  performance  in 

these  trials.  This  result  suggests  that  ballistic  motion  accounted  for  the  most 
significant  part  of  movement  times.  An  interesting  side  note  is  that  in  a  time 
delay  study  currently  being  completed.  Fixture  6  was  found  to  provide 
significant  improvement  over  Fixture  3  when  communication  delays  were 
introduced  into  the  system.  This  makes  sense  because  fine  position  becomes 
more  difficult  when  time  delays  are  present.  These  results  will  be  presented 
formally  in  a  companion  paper  addressing  time  delay  issues. 

Virtual  Fixture  7 

As  shown  in  Figure  6,  virtual  Fixture  7  is  identical  to  Fixture  1  with  the 
addition  of  field-like  auditory  feedback.  The  sonic  information  is  modeled  as  a 
series  of  parallel  planes  such  that  crossing  of  each  plane  results  in  a  tone  of  a 
given  frequency.  As  planes  are  crossed  from  left  to  right,  the  frequency 

associated  with  each  plane  increases.  Both  the  density  of  auditory  pulses  and 
the  pitch  of  each  pulse  are  useful  cues.  The  result  is  a  texture-like  sonic  field 
which  provides  rich  position,  velocity,  and  acceleration  feedback  to  the 
operator.  As  shown  in  Tables  5,  10  and  11,  the  mean  movement  times  for  peg 
insertions  performed  with  the  aid  of  virtual  Fixture  7  were  20%  faster  than  the 


Fixture  1  case  and  42%  faster  than  with  the  no-fixture  case.  Comparing 
Indexes  of  Performance  (Ip)  as  shown  in  Tables  8,  12  and  13,  the  human 
information  processing  capacity  when  using  virtual  Fixture  7  was  23%  greater 
than  the  Fixture  1  case  and  52%  greater  than  with  no  fixture  at  all.  The  over 

20%  improvement  in  performance  when  using  Fixture  7  over  Fixture  1  (which 

differ  only  by  the  addition  of  auditory  information  in  7)  strongly  suggests  that 
the  use  of  multiple  sensory  modalities  increases  information  processing 
capacity  and  is  thus  a  powerful  tool  in  fixture  design.  It  further  suggests  that 
overlaying  an  auditory  gradient  field  on  top  of  the  workspace  to  provide 
position,  velocity,  and  acceleration  cues  is  a  simple  means  of  enhancing 
operator  performance.  Post  testing  subject  interviews  revealed  that  most 
subjects  were  unaware  that  their  performance  increased  with  the  additional 
auditory  information,  but  all  reported  having  a  better  sense  of  the  shape  of 

their  peg  insertion  trajectories. 

Virtual  Fixture  8 

As  shown  in  Figure  6,  Fixture  8  is  identical  to  Fixture  6  with  the  addition 
of  an  auditory  compliant  surface  modeled  such  that  compression  of  the  surface 
corresponds  to  increasing  pitch  of  the  auditory  feedback.  The  surface  is 
positioned  such  that  first  contact  occurs  just  before  the  peg  should  be  inserted 
into  the  target  hole.  This  auditory  surface  provides  a  well  positioned  localizing 
reference  to  act  as  a  warning  signal  to  indicate  that  the  peg  has  traveled  far 
enough  and  now  should  be  inserted.  As  shown  in  Tables  5,  10  and  11,  the  mean 
movement  times  for  peg  insertions  performed  with  the  aid  of  virtual  Fixture  8 
were  12%  faster  than  the  Fixture  6  case  and  53%  faster  than  with  the  no¬ 
fixture  case.  Comparing  Indexes  of  Performance  (Ip)  as  shown  in  Tables  8,  12 
and  13,  the  human  information  processing  capacity  when  using  virtual 
Fixture  8  was  13%  greater  than  the  Fixture  6  case  and  70%  greater  than  with 

no-fixture  at  all.  The  over  12%  improvement  in  performance  when  using 
Fixture  8  over  Fixture  6  (which  differ  only  by  the  addition  of  auditory 
information  in  8)  suggests  that  the  use  of  the  additional  modality  was  useful  in 
increasing  information  processing  capacity.  It  should  also  be  noted  that  of  all 
the  fixtures  tested,  the  use  of  Fixture  8  resulted  in  the  greatest  improvement  in 
movement  times  and  Index  of  Performance  over  the  no-fixture  case.  This 
suggests  that  the  use  of  multiple  modalities  in  fixture  design  is  a  powerful 
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perceptual  aid  and  should  be  expanded  to  visual  and  tactile  information  as  well. 
Post  testing  interviews  revealed  that  the  use  of  the  auditory  fixtures  caused 
subjects  to  alter  their  conceptualization  of  the  task  such  that  a  successful  trial 
no  longer  Just  looked  and  felt  a  certain  way,  but  also  sounded  a  certain  way. 

We  can  further  compare  the  advantage  of  using  multiple  modalities  in 
fixture  design  by  analyzing  the  slopes  of  the  Fitts'  Law  curve  relating 
movement  times  to  Index  of  Difficulty  for  each  fixture.  The  inverse  of  the 
slope  of  the  Fitts'  Law  curve,  computed  by  linear  regression  and  listed  in  Table 
9,  has  units  (bits/sec)  and  is  an  accepted  measure  of  overall  information 
processing  capacity  of  the  operator.  This  information  is  also  represented  in 
graphical  form  in  Figure  8.  Comparing  each  fixture  tested,  the  two  fixtures 
whose  use  resulted  in  the  greatest  overall  increase  in  information  processing 
capacity  were  Fixtures  7  and  8,  the  only  two  fixtures  to  introduce  auditory 
information.  This  analysis  further  supports  the  idea  that  using  virtual 

fixtures  to  introduce  abstract  sensory  information  from  multiple  modalities  is 
a  viable  method  of  enhancing  operator  performance. 
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CONCLUSIONS 


The  results  of  this  study  confirm  that  overlaying  abstract  sensory 
information  in  the  form  of  virtual  fixtures  on  top  of  the  sensory  feedback  from 
a  remote  environment  can  greatly  enhance  performance  in  telemanipulation 
tasks.  Virtual  fixtures  composed  of  simple  combinations  of  rigid  impedance 

surfaces  and  abstract  auditory  information  increased  operator  performance  by 
up  to  70%  in  a  standard  peg  insertion  task.  It  should  be  noted  that  a  peg 
insertion  task  was  chosen  for  this  study  because  it  offered  an  effective  means  of 
quantifying  operator  performance  and  not  because  it  was  thought  to  be  a 
particularly  good  application  for  virtual  fixtures.  Although  virtual  fixtures 
proved  to  be  an  effective  means  of  improving  peg  insertion  performance,  it  is 
likely  that  the  fixtures  would  have  had  a  much  greater  impact  upon  the 
performance  of  a  more  challenging  telemanipulation  task. 

Because  effective  virtual  fixtures  were  developed  from  very  basic 

elements  like  rigid  impedance  surfaces  and  simple  gradient  fields,  the 

development  of  an  interactive  perceptual  workstation  that  allows  an  operator  to 
build  virtual  fixtures  from  basic  building  blocks  seems  like  a  feasible  endeavor. 
Such  an  interactive  workstation  could  allow  a  teleoperator  to  develop  powerful 
virtual  fixtures  upon  first  encounter  with  an  unfamiliar  task  in  an 
unstructured  environment.  Such  a  workstation  might  also  allow  an  operator  to 

quickly  modify  a  fixture  as  task  conditions  change. 

Analysis  of  some  basic  perceptual  elements  tested  in  this  study  suggests 
that  virtual  fixtures  enhance  operator  performance  in  the  following  ways: 

1.  Virtual  fixtures  simplify  the  operator’s  interaction  with  the  remote 
workspace  by  restricting  unnecessary  kinematic  freedoms  and  preventing 
unwanted  actions  from  occurring.  This  simplification  limits  the  alternatives 
the  operator  has  to  consider  and  reduces  the  information  that  needs  to  be 
processed.  This  simplification  also  allows  the  operator  to  relax  criteria  for 
success  and  failure  by  eliminating  some  modes  of  failure  and  by  providing 
assistance  in  achieving  some  aspects  of  a  successful  task  completion. 
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2.  Virtual  fixtures  alter  the  operator's  conceptualization  of  the  remote 
task  by  introducing  abstract  sensory  information  into  the  workspace.  Without 
the  aid  of  any  virtual  fixturing,  the  remote  peg  insertion  task  was  primarily  a 
visual  operation.  Thus,  the  operator  was  likely  to  conceptualize  the  peg 
insertion  task  as  a  manual  procedure  that  “looks  a  certain  way.’’  With  virtual 
fixtures,  additional  haptic  and  auditory  information  is  provided  to  the  operator. 
This  additional  sensory  information  allows  the  operator  to  conceptualize  the  peg 
insertion  task  as  not  just  “looking  a  certain  way’’  but  also  as  “feeling  a  certain 
way’’  and  “sounding  a  certain  way." 

3.  Virtual  fixtures  were  shown  to  provide  a  localizing  reference  to  the 
remote  worksite  by  introducing  haptic  and  auditory  cues  that  can  be  coupled  to 
the  operator’s  kinesthetic  sense  of  workspace  geometry.  This  was  demonstrated 
dramatically  by  test  Fixture  5  which  allowed  a  20%  performance  improvement 
by  providing  haptic  localizing  information  to  the  operator's  unused  hand. 

4.  Virtual  fixtures  can  displace  the  burden  from  taxed  sensory  modalities 
by  providing  information  through  alternative  sensory  pathways.  Without  the 
aid  of  virtual  fixtures,  the  peg  insertion  task  primarily  taxed  the  visual 
modality.  With  the  fixtures,  haptic  and  auditory  sensations  could  also  be  relied 
upon.  Had  this  task  required  additional  use  of  the  visual  modality  for  scanning 
the  scene  or  monitoring  some  other  aspect  of  the  task,  the  beneficial  effect  of 
displacing  the  burden  from  the  visual  modality  would  likely  have  been  even 
more  pronounced. 
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